Search for: All records

Does the dominant approach to learn representations (as a side effect of optimizing an expected cost for a single training distribution) remain a good approach when we are dealing with multiple distributions? Our thesis is that such scenarios are better served by representations that are richer than those obtained with a single optimization episode. We support this thesis with simple theoretical arguments and with experiments utilizing an apparently na\"ıve ensembling technique: concatenating the representations obtained from multiple training episodes using the same data, model, algorithm, and hyper-parameters, but different random seeds. These independently trained networks perform similarly. Yet, in a number of scenarios involving new distributions, the concatenated representation performs substantially better than an equivalently sized network trained with a single training run. This proves that the representations constructed by multiple training episodes are in fact different. Although their concatenation carries little additional information about the training task under the training distribution, it becomes substantially more informative when tasks or distributions change. Meanwhile, a single training episode is unlikely to yield such a redundant representation because the optimization process has no reason to accumulate features that do not incrementally improve the training performance. 

There often is a dilemma between ease of optimization and robust out-of-distribution (OoD) generalization. For instance, many OoD methods rely on penalty terms whose optimization is challenging. They are either too strong to optimize reliably or too weak to achieve their goals. We propose to initialize the networks with a rich representation containing a palette of potentially useful features, ready to be used by even simple models. On the one hand, a rich representation provides a good initialization for the optimizer. On the other hand, it also provides an inductive bias that helps OoD generalization. Such a representation is constructed with the Rich Feature Construction (RFC) algorithm, also called the Bonsai algorithm, which consists of a succession of training episodes. During discovery episodes, we craft a multi-objective optimization criterion and its associated datasets in a manner that prevents the network from using the features constructed in the previous iterations. During synthesis episodes, we use knowledge distillation to force the network to simultaneously represent all the previously discovered features. Initializing the networks with Bonsai representations consistently helps six OoD methods achieve top performance on ColoredMNIST benchmark. The same technique substantially outperforms comparable results on the Wilds Camelyon17 task, eliminates the high result variance that plagues other methods, and makes hyperparameter tuning and model selection more reliable. 

Measurements are presented of the cross-section for the central exclusive production ofJ/\psi\to\mu^+\mu^- $J/\psi \to {\mu }^{+}{\mu }^{-}$and\psi(2S)\to\mu^+\mu^- $\psi \left(2S\right)\to {\mu }^{+}{\mu }^{-}$processes in proton-proton collisions at\sqrt{s} = 13 \ \mathrm{TeV} $\sqrt{s}=13TeV$with 2016–2018 data. They are performed by requiring both muons to be in the LHCb acceptance (with pseudorapidity2<\eta_{\mu^±} < 4.5 $2<{\eta }_{{\mu }^{\pm }}<4.5$) and mesons in the rapidity range2.0 < y < 4.5 $2.0<y<4.5$. The integrated cross-section results are\sigma_{J/\psi\to\mu^+\mu^-}(2.0 ${\sigma }_{J/\psi \to {\mu }^{+}{\mu }^{-}}(2.0<y_{J/\psi }<4.5,2.0<{\eta }_{{\mu }^{\pm }}<4.5)=400\pm 2\pm 5\pm 12pb,{\sigma }_{\psi \left(2S\right)\to {\mu }^{+}{\mu }^{-}}(2.0<y_{\psi \left(2S\right)}<4.5,2.0<{\eta }_{{\mu }^{\pm }}<4.5)=9.40\pm 0.15\pm 0.13\pm 0.27pb,$where the uncertainties are statistical, systematic and due to the luminosity determination. In addition, a measurement of the ratio of\psi(2S) $\psi \left(2S\right)$andJ/\psi $J/\psi$cross-sections, at an average photon-proton centre-of-mass energy of1\ \mathrm{TeV} $1TeV$, is performed, giving$ = 0.1763 ± 0.0029 ± 0.0008 ± 0.0039,$$ where the first uncertainty is statistical, the second systematic and the third due to the knowledge of the involved branching fractions. For the first time, the dependence of theJ/\psi$ $J/\psi$and\psi(2S) $\psi \left(2S\right)$cross-sections on the total transverse momentum transfer is determined inpp $pp$collisions and is found consistent with the behaviour observed in electron-proton collisions.